Stateless neural meta-learning using second-order gradients

Abstract Meta-learning can be used to learn a good prior that facilitates quick learning; two popular approaches are MAML and the meta-learner LSTM. These methods represent important different in meta-learning. In this work, we study formally show LSTM subsumes MAML, although which is sense less general, outperforms other. We suggest reason for surprising performance gap related second-order gradients. construct new algorithm (named TURTLE) gain more insight into importance of TURTLE simpler than yet expressive both techniques at few-shot sine wave regression 50% tested image classification settings (without any additional hyperparameter tuning) competitive otherwise, computational cost comparable MAML. find gradients also significantly increase accuracy When was introduced, one its remarkable features use Subsequent work focused on cheaper first-order approximations. On basis our findings, argue attention